Rethinking detection based table structure recognition for visually rich document images

As discussed, this study focuses on the detection-based solution for the TSR task, and many issues are limiting detection-based TSR solutions. Therefore, the research objectives of this study are three-fold: (1) This study explores and reveals the underlying reasons and factors impeding the performance of existing detection-based TSR solutions. (2) Based on our analysis and findings, this study builds a state-of-the-art detection-based solution for the TSR task regarding COCO metrics and structural-only TEDS. (3) This study discusses and summarizes the critical design aspects for the success of a detection-based TSR model based on the observations from the experimental results and analysis.

The contributions of this study are four-fold:

Section 2 discusses related studies, including studies in Object Detection and Table Structure Recognition. Section 3 explores and discusses the reasons that hinder the performance of detection-based TSR models. Section 4 describes our proposed solution based on our analysis and findings. Section 5 shows the experiment settings and experimental results. Section 6 discusses the design aspects of the proposed method. At last, we draw our conclusion and possible directions in Section 7.

Object Detection is a fundamental task that has been widely discussed in many studies. Since deep models have become the dominant solutions, we only discuss popular deep learning based models in this section. Based on different design perspectives, popular detection models can be categorized in different ways. One popular categorization of detectors based on the number of regression steps is to classify them into one- and two-stage detectors. Two-stage models, such as Cascade R-CNN (Cai & Vasconcelos, 2018), usually use a Region Proposal Network (RPN) to generate region proposals first and then feed the region proposals to the well-designed model to classify and regress the proposals. In the RPN network, one key parameter is the aspect ratio, which defines the height/width ratio when generating anchor boxes. Suitable aspect ratios are often close to the target objects’ height/width ratio, making the regression task easier and improving the model performance. In contrast, popular one-stage models, such as FCOS (Tian, Shen, Chen, & He, 2019), YOLO series models (Bacea and Oniga, 2023, Li, Li et al., 2022, Wang, Bochkovskiy et al., 2023), integrate the region proposal generation and other regression and classification components into a single network. For example, YOLO series models divide the images into grids first, then classify the class of grid cells and directly predict the bounding boxes and their confidences. The simple design of one-stage detectors leads to faster training and inference time compared with two-stage detectors.

On the other hand, some studies (Sun, Zhang et al., 2021, Zhang et al., 2023) categorize the popular detectors from the perspective of Non-maximum Suppression (NMS), which is widely used to reduce the redundant predictions from the detectors. From this perspective, popular detectors can be categorized into end-to-end and none end-to-end models based on whether NMS is needed. DETR (Carion et al., 2020) is a typical end-to-end detector introducing transformer architecture (Vaswani et al., 2017), set prediction loss, and one-to-one label assignment to the object detection problem. Sparse R-CNN (Sun, Zhang et al., 2021) further refactors the DETR model and proposes to use sparse learnable regional proposals to replace dense regional proposals and utilize a dynamic instance interactive head to regress and classify the proposals in an iterative manner. Study (Sun, Jiang et al., 2021) analyzes the success of end-to-end detectors and argues that the one-to-one label assignment method in end-to-end detectors contributes to the success of the end-to-end models but is not sufficient to fully remove the NMS from the pipeline. This study further points out that the classification cost in the matching cost when applying one-to-one label assignment plays a key role in the success of these end-to-end models. Study (Zhang et al., 2023) further analyzes combinations of the label assignment methods and queries and argues that sparse queries with one-to-one label assignment can degrade the recall, and dense queries with one-to-one label assignment are hard to optimize. To address these issues, study (Zhang et al., 2023) proposes a dense distinct queries (DDQ) method to select distinct queries from dense queries using a class-agnostic NMS, achieving promising precision and recall. SQR (Chen et al., 2023) points out that the stages in DETR series detectors have different unbalanced responsibilities and proposes to collect and select intermediate queries for subsequent stages. It is worth mentioning that these end-to-end detector can easily be extended to none end-to-end solutions by adapting many-to-one label assignments (Hong et al., 2022) and NMS. In this study, we refer to models using transformer architecture, set prediction loss, and their variations as transformer-based detection models, such as DETR, Sparse R-CNN and Deformable-DETR (Zhu et al., 2021).

There have been many studies (Adiga et al., 2019, Chi et al., 2019, Liu, Li et al., 2022, Schreiber et al., 2017, Xiao, Akkaya et al., 2022, Zheng et al., 2021) discussing the TSR problem in recent years. As mentioned in Section 1, we can roughly categorize these solutions into image-to-sequence, detection-based, and graph-based models. Image-to-sequence based models usually define the ground truth as structured sequences, such as HTML sequences, built on the transformer architecture (Vaswani et al., 2017), and follow an encoder–decoder architecture. For instance, TableMaster (Ye et al., 2021) is a typical image-to-sequence based model that can generate HTML sequences. More specifically, TableMaster follows the architecture of MASTER (Lu et al., 2021), which is originally designed for the scene text generation following the transformer architecture (Vaswani et al., 2017), and further improved the encoder part by introducing a Multi-Aspect Global Context Attention. Besides, TableMaster has two branches designed for the HTML sequence generation and bounding box regression. Similarly, MTL-TabNet (Ly & Takasu, 2023) also follows the encoder–decoder architecture but contains three decoders for the cell box regression, cell content recognition, and HTML sequence generation, respectively. DRCC (Shen et al., 2023) argues that the error accumulation problem degrades the performance of image-to-sequence TSR models, especially when the input image is large. Therefore, DRCC proposes a two-step decoder architecture, which first decodes the input image into rows and then decodes the rows in cell sequences. VAST (Huang et al., 2023) pays more attention to the imprecise bounding boxes of table cells and proposes a Coordinate Sequence Decoder to improve the model’s ability to generate accurate bounding boxes and introduces a visual-alignment loss to align the visual and structural information. To sum up, this type of method is usually based on the encoder–decoder architecture and can be trained end-to-end without using post-processing methods. Since the ground truth sequences used in image-to-sequence models usually contain information regarding spanning cells and header cells, these models can handle complex structures with spanning cells and identify header cells.

On the other hand, detection-based models usually define the problem as detecting different table components and applying a post-processing method to reconstruct table structures. DeepTabStR (Siddiqui et al., 2019) proposes to detect columns and rows to obtain the table cells. However, DeepTabStR ignores the row/column-span in the tables, which means that it cannot recover the hierarchical structures of tables. TableStrRec (Fernandes et al., 2023) extends the DeepTabStR, defining four types of table components: regular columns, irregular columns, regular rows, and irregular rows. Then, the spanning cells across multiple columns can be inferred from the difference between the regular and irregular columns when they are overlapped, and the spanning cells across multiple rows can be inferred from regular and irregular rows similarly. PubTables1M (Smock et al., 2022) is another typical detection-based approach that defines six table components: table, column, row, spanning cell, Projected Row Header, and Column Header, in which Projected Row Header and Column Header are for the function analysis, and other components can be used to reconstruct the complex table structure. Among these formulations, only the problem formulation of PubTables1M can provide as much information as image-to-sequence models because it can provide header cell information and reconstruct the complex table structure. Besides, these detection-based models need an extra deterministic rule-based post-processing method to infer the table structure from detected table components, meaning they are not end-to-end.

At last, graph-based methods usually apply either detection or segmentation methods to obtain the locations of table cells and further build the relation among table cells. For instance, TGRNet (Xue, Yu, Wang, Tao, & Li, 2021) formulates the cell location detection and cell logical location prediction jointly in a multi-task architecture, which is modularized by a segmentation based method and graph convolutional network (GCN), respectively. Similarly, TSRNet (Li, Yin et al., 2022) proposes a unified GNN-based approach modeling table detection and table structure recognition tasks together. More specifically, TSRNet also employs a semantic segmentation module to extract primitive regions, then applies k-nearest neighbors and line-of-sight neighbors to construct the graph and further classify the graph nodes and edges to filter the noise regions, merge, and build relations. In contrast, LGPMA (Qiao et al., 2021) proposes a Local Pyramid Mask Alignment Module and Global Pyramid Mask Alignment Module to localize table cells, which are formulated as detection and segmentation problems and can be implemented by MaskR-CNN (He, Gkioxari, Dollár, & Girshick, 2017). To construct the structure of the table, LGPMA further proposes a pipeline of cell matching, empty cell searching, and empty cell merging using the Maximum Clique Search algorithm and rule-based methods. Besides building graph explicitly, some studies (Nguyen et al., 2023, Tensmeyer et al., 2019, Zhang et al., 2022) predict the table grids or separators first, and then merge grid elements, which are also treating grid elements as graph nodes. SPLERGE (Tensmeyer et al., 2019) is a typical method following this strategy consisting of a Split Model and Merge Model, in which the Split Model consists of a Row Projection Network and a Columns Projection Network to obtain the table grid, and the Merge Model is used to merge the grid cells. Similarly, SEM (Zhang et al., 2022) employs a segmentation model to segment columns and rows and generate the table grid with a post-processing method. After the table grid is obtained, SEM introduces an Embedder network to extract and fuse the features from textual and visual modalities. A Merger network takes the fused features from Embedder as inputs to merge the grid elements. TSRFormer-DQ-DETR (Wang, Lin et al., 2023) leverages a DETR (Carion et al., 2020) based separation line prediction model, termed DQ-DETR, to predict the reference points on separation lines, followed by a Relation Network based cell Merging module to merge grid elements. RobustTabNet (Ma et al., 2023) employs a spatial network to predict Row and Column separation lines and further introduces a Grid CNN module to merge and build relations of table cells. Since these graph-based models identify the graph nodes first, defining a cell-type classification task is necessary if they want to provide information regarding header cells.

Since most existing detection-based TSR models are based on two-stage and transformer-based detectors, we use Cascade R-CNN (Cai & Vasconcelos, 2018) and Sparse R-CNN (Sun, Zhang et al., 2021) as two examples of these two types of detectors and briefly review their critical designs in this section.

As aforementioned in Section 1, there have been many detection-based solutions (Fernandes et al., 2023, Hashmi et al., 2021, Siddiqui et al., 2019, Smock et al., 2022, Xiao, Akkaya et al., 2022) with different problem formulations that either oversimplified the TSR task or ignored its multi-label characteristic. More specifically, following image-to-sequence TSR models, a detection-based TSR model should be able to fully reconstruct the structure of both regular and spanning table cells, as well as provide information regarding header cells. However, studies (Hashmi et al., 2021, Siddiqui et al., 2019) formulate the problem as only detecting columns and rows, making them impossible to deal with spanning cells and identify header cells. TableStrRec (Fernandes et al., 2023) further extend the formulation by defining regular column, regular row, irregular column, and irregular row so that the spanning cell can be inferred from these four types of components, as shown in Fig. 4(a). But still, this formulation cannot provide information regarding header cells, which is still over-simplified the TSR task. Study (Xiao, Akkaya et al., 2022) simplifies the formulation of in PubTables1M (Smock et al., 2022), directly detecting table, column, row, and spanning cell, as shown in Fig. 4(b), which is another formulation ignoring header cells’ information. Besides, these three formulations treat the Projected Row Header as a regular row, resulting in over-simplified table structures. By contrast, PubTables1M (Smock et al., 2022) defines six types of components, including Table, Column, Row, Spanning Cell, Column Header, and Projected Row Header, as shown in Fig. 4, Fig. 4, which can provide as much structure information as image-to-sequence TSR models. However, this formulation does not consider that some Column Headers and Projected Row Headers can share identical bounding boxes with corresponding Rows. For example, as shown in Fig. 4(c), the bounding boxes of the two Projected Row Headers can also be classified as Rows. Similarly, as shown in Fig. 4(d), the Column Header’s bounding box is also the Row’s bounding box. Therefore, the problem definition of study (Smock et al., 2022) is a multi-label detection problem, which is must be considered when we choose and design detection models. It is worth mentioning that all these problem formulations use extracted table images as inputs. Even though many studies (Prasad et al., 2020, Xiao et al., 2023b, Xiao et al., 2023c) have achieved very promising performance on the Table Detection (TD) task, it is still difficult to guarantee that all the table content can be fully included in the detection results. Therefore, in practice, the detected bounding boxes of tables from the TD model are often padded with extra pixels, making it necessary to define a Table component for TSR. We summarize these problem formulations in Table 1 focusing on the detection targets, evaluation metrics, outputs after post-processing, and potential issues. The formulation in this study can avoid the issues of other formulations, including the information loss and the multi-label detection formulation.

As aforementioned in Section 3.1, the parameters of generating anchor boxes in the RPN play a key role in two-stage detection models, while transformer-based detection models, such as DETR and Sparse R-CNN, use learnable queries or proposals without the need to tune the RPN. If we choose two-stage detection models, such as Cascade R-CNN which is used in TableStrRec (Fernandes et al., 2023), we need to identify the difference between the TSR detection problem and widely discussed common object detection problem, because the default settings of detection frameworks, such as Detectron2 (Wu, Kirillov, Massa, Lo, & Girshick, 2019) and MMDetection (Chen et al., 2019) are often tuned on COCO (Lin et al., 2014) dataset. Therefore, we compare the statistics of the COCO dataset with a popular TSR dataset, FinTabNet (Zheng et al., 2021), regarding the number of objects in each image and the aspect ratios of objects. More specifically, the COCO training set contains 118 287 images and 860 001 target objects, resulting in an average of 7.27 objects in each image, while the FinTabNet training set contains 78 537 images, 1 628 298 target objects, resulting in an average of 20.73 objects in each image. Besides, the aspect ratios of objects in these two dataset are also very different, as shown in Fig. 5. The vast majority of target objects in the COCO training set have aspect ratios between 1 and 10, while objects in the FinTabNet training set have much larger aspect ratios. Therefore, we need to consider these differences when tuning the parameters of RPN if we apply a two-stage object detection model for the TSR task, such as increasing the number of region proposals and adjusting the aspect ratios of anchor boxes. On the other hand, transformer-based detection models, such as Sparse R-CNN and DETR, can alleviate the issues caused by these differences intrinsically because they use learnable queries (learnable proposals) instead of an RPN, as discussed in Section 3.1. However, increasing the number of learnable queries for each image might be also useful for transformer-based detection models because TSR datasets contain more objects than common object detection datasets.

As mentioned in Sections 1 Introduction, 2 Related work, detection-based TSR models need a deterministic rule-based post-processing method to transform the detected table components into structured sequences. Existing studies (Fernandes et al., 2023, Hashmi et al., 2021, Siddiqui et al., 2019, Xiao, Akkaya et al., 2022) usually use the detection performance to evaluate the model performance before applying the post-processing method. However, the detection metrics are not aligned with cell-level TSR metrics. We use COCO (Lin et al., 2014) and TEDS (Zhong et al., 2020) metrics as examples for further analysis in this section. The COCO metrics employ mean Average Precision (mAP) to evaluate the model performance, which can be defined by Eq. (6) where $N$, $precisio{n}_i\left(r\right)$ and $dr$ in Eq. (6) are the number of classes, and the precision at a given recall level $r$ for class $i$. In practice, the precision–recall curves in COCO metrics are computed for each class at a series of IoU thresholds, and the integral of $precisio{n}_i\left(r\right)$ often is approximated by the discrete sum. The IoU score can be defined by Eq. (7), where $A\cap B$, $A\cup B$ are the intersection and union of bounding boxes $A$ and $B$. In practice, in many studies, mAP is represented by AP and calculated by averaging the mean precision scores of all categories at IoU thresholds from 0.5 to 0.95 with 0.05 intervals. AP50, AP75 are the mean precision scores of all categorizes at IoU thresholds 0.5 and 0.75, respectively. Therefore, COCO metrics are IoU-based evaluation metrics. By contrast, TEDS can be defined by Eq. (8), where EditDist is the tree-edit distance, and $T$ is the number of nodes in the tree, meaning that TEDS is not correlated with IoU scores. (6)$mAP=\frac{1}{N}\sum _{i=1}^N\left.{\int }_0^1{\text{precision}}_i\left(r\right)dr\right)$ (7)$IoU=\frac{A\cap B}{A\cup B}$ (8)$TEDS(T_a,T_b)=1-\frac{EditDist(T_a,T_b)}{max(\mid T_a\mid ,\mid T_b\mid )}$

As mentioned in Section 1, deformable convolution (Dai et al., 2017) has been applied in detection-based TSR (Fernandes et al., 2023, Siddiqui et al., 2019) and other related solutions (Mondal et al., 2023, Siddiqui et al., 2018), demonstrating its effectiveness in improving detection performance. Deformable convolution uses a learnable grid offset to sample the grid points from the feature map and then apply the convolution operation to the sampled grid points, as defined by Eq. (9), (9)${\mathbf{z}}_{p_0}=\sum _{p_n\in R}w\left(p_n\right)x(p_0+p_n+\Delta p_n)$where $p_0$ is the location on the output feature map $\mathbf{z}$, $p_n$ is the $n$th grid point in grid $R$, and $\Delta p_n$ is the $n$th learnable offset. Since the offset $\Delta p_n$ applied to the deformable convolution is usually obtained by a regular convolution with small kernels, such as a 3 * 3 kernel, it can only improve the local feature instead of building long-range dependencies. However, building the long-range dependencies for the TSR task is important because of the characteristics of table components. More specifically, different parts of a single table component are often sparsely distributed across the table instead of a single area of compact pixels like common objects. Fig. 7 shows a sample with its Row annotations. Taking the first Row as an example, as shown in Fig. 7, it mainly contains three parts, which are distributed sparsely, and there is a large space between the first part and the second part, even they all belong to a single target component. Therefore, it is important to build long-range dependencies together with improving local features, such as applying deformable convolution. And over-optimized local features, such as merely applying deformable convolution might degrade the performance regarding the TEDS.

As mentioned in Sections 3.1 Preliminaries, 3.2 Rethinking problem formulations, the definition of PubTables1M (Smock et al., 2022) can provide as much information as other types of solutions and is a multi-label detection problem, which is challenging for two-stage detectors. Therefore, we follow PubTables1M to define six table components: Table, Column, Row, Spanning Cell, Projected Row Header, and Column Header, and transform the formulation into a single-class detection problem. More specifically, we remove the Rows that share their bounding boxes with the Projected Row Header, as shown in Fig. 8(a), and use a Pseudo Class to replace the Rows and Column Headers when they share identical bounding boxes, as shown in Fig. 8(b). It is worth mentioning that only the Row, Projected Row Header, and Column Header are shown because the Table, Column, and Spanning Cell are the same as PubTables1M. These two samples are also in Fig. 4, Fig. 4, which show their original definition in PubTables1M.

Formally, the ground truth $Y$ in PubTable1M’s definition for each image is a set of tuples containing bounding boxes and their corresponding labels, as defined by Eq. (10), where $b_i,c_i$ are the $i$th bounding box and its class, and values from $0$ to $5$ are the defined Table, Column, Row, Spanning Cell, Projected Row Header and Column Header, respectively. (10)$Y={\left\{(b_i,c_i)\right\}}_{i=1}^N,c_i\in \{0,1,2,3,4,5\},\forall i\ne j,(c_i\ne c_j)\wedge ((b_i=b_j)\vee (b_i\ne b_j))$

By contrast, in this study, considering the observation that the defined Projected Row Headers are all Rows at the same time, we only keep the Projected Row Headers samples during the training. Since some Column Headers can share identical bounding boxes with corresponding Rows, we derive a pseudo class for these overlapped samples and remove the original overlapped samples. Therefore, during the training stage, we refactor the ground truth for each image to the regular single-label classification, as defined by Eq. (11), where values $0$ to $6$ are the Table, Column, Row, Spanning Cell, Projected Row Header, Column Header and the Pseudo Class, respectively. During the testing stage, the results of Project Column Header are duplicated once to generate their corresponding prediction Rows, and the results of the pseudo-class are duplicated twice to generate the corresponding prediction Rows and Headers, so that we can still follow the formulation defined by Eq. (10) to evaluate the model performance. Notably, we only apply this problem formulation to our tailored Cascade R-CNN model, and all other detection benchmark models are following the formulation of PubTables1M. (11)$Y={\left\{(b_i,c_i)\right\}}_{i=1}^N,c_i\in \{0,1,2,3,4,5,6\},\forall i\ne j,(c_i\ne c_j)\wedge (b_i\ne b_j)$

As mentioned in Sections 3.1 Preliminaries, 3.3 Revisiting region proposal generation, regional proposal generation is a critical step in two-stage detectors, which need to be carefully considered for the TSR problem. Therefore, we adjust Aspect Ratios and increase the number of generated regional proposals for our tailored model. More specifically, aspect ratios control the shape of the generated anchor boxes. Popular implementations of Cascade R-CNN, such as Detectron2 (Wu et al., 2019), usually use 0.5, 1.0, and 2.0 as default values, which can work well for detecting common objects, such as the objects in COCO (Lin et al., 2014) dataset. However, in the context of TSR, the range of aspect ratios is much larger because of the shape of the table components, as discussed in Section 3.3. Without proposing fancy new modules to select suitable values, we simply select the values based on the statistics of the training sets. Taking the FinTabNet dataset as an example, the aspect ratios of the defined components are shown in Fig. 5. The maximum value is 140, far larger than the popular choices in common object detection. Besides, the majority of aspect ratios in Fig. 5 are in the range between 1 and 60. Therefore, we extend this parameter for our proposed model as [0.0125, 0.025, 0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0, 16, 40, 80]. A further detailed parameter table is provided in Section 5. It is worth mentioning that when an aspect ratio is less than 1, its multiplicative inverse is applied to count the number of aspect ratios in Fig. 5. We also did not further fine-tune this parameter through validation, which means that they might not be optimal. But this parameter has improved the model performance by around 2.7% as shown in Section 5.4. Besides, since increasing the number of proposals has been applied in existing studies (Fernandes et al., 2023), and demonstrated its effectiveness, we increase it for both the base Cascade R-CNN and our proposed model.

As discussed in Section 3.5, building long-range dependencies for detecting the defined components is important. Inspired by the recent studies using large convolution kernels (Ding et al., 2022, Liu, Mao et al., 2022), we introduce a Spatial Attention Module for our solution, whose architecture is shown in Fig. 9. For the design of the Spatial Attention Module, we use a similar architecture with MSCA (Guo et al., 2022) containing multiple branches and large kernel convolutions and use spatial and depthwise separable convolution (Chollet, 2017, Howard et al., 2017) to reduce the number of parameters. More specifically, for the spatial separable convolution, we use a pair of $7\ast 1$ and $1\ast 7$ kernels to replace a typical $7\ast 7$, use the pair of $11\ast 1$ and $1\ast 11$, and the pair of 21 × 1 and 1 × 21 to replace $11\ast 11$ and $21\ast 21$ kernels, respectively. For the depthwise separable convolution, we applied the convolution on each channel of the feature maps independently. Then, the outputs of the three branches are concatenated together as the input of a convolution layer with 1 × 1 kernel to make the channel dimension the same as the inputs. The proposed Spatial Attention Module can be easily inserted into the Backbone Network between two blocks because they do not change the feature shapes. For example, for a typical backbone network implemented by ResNet (He, Zhang, Ren, & Sun, 2016) containing a STEM block and four Residual Blocks, as shown in Fig. 9, the Spatial Attention Module can be inserted after the last three Residual Blocks to generate the spatial attention, then the spatial attention can be applied to the original outputs of each Residual Block by Element-wise Multiplication. It is worth mentioning that the Spatial Attention Module shown in Fig. 9 have independent trainable parameters, and all the feature maps are padded correspondingly to keep the size of the feature maps.

We utilize four datasets in this work, including SciTSR (Chi et al., 2019), FinTabNet (Zheng et al., 2021), PubTabNet (Zhong et al., 2020) and PubTables1M (Smock et al., 2022). As discussed in the study (Smock, Pesala, & Abraham, 2023), FinTabNet and SciTSR datasets contain noise annotations that harm the model performance. Therefore, we use their cleaned versions proposed in the study in Smock et al. (2023). Each image sample in these four datasets contains only a table with extra padding pixels to ensure the entire table is extracted. The SicTSR dataset is collected from academic publications containing 7453, 1034, and 2134 samples for training, validation, and testing. PubTables1M dataset is a large-scale dataset for the TSR problem collected from the PMCOA corpus, containing 758 849 training samples, 94 959 validation samples, and 93 834 testing samples. Since the PubTabNet dataset does not provide original PDF files, we cannot process it to make detection annotations. Besides, its testing is not publicly available. Therefore, we use its validation set to evaluate the model trained with the PubTable1M dataset. Following the study in Smock et al. (2023), we use the code base in Smock and Pesala (2021) to process the datasets and align the formats of these datasets. FinTabNet is also a large dataset widely used for the TSR problem, containing 78 537, 9289, and 9650 samples for training, testing, and validation. FinTabNet is collected from the annual reports of companies, making its data source different from the other datasets. Table 2 summarizes the datasets used in this study for the model evaluation.

For the TSR metric, we choose structure-only Tree-Edit-Distance-Based Similarity(TEDS) (Zhong et al., 2020), which is firstly introduced in the study to overcome the drawbacks of adjacency relation metrics, and can be defined as Eq. (8) as discussed in Section 3.4. We use structure-only TEDS in the study, which only considers the HTML tags without extracting their contents to avoid the influence of OCR tools. The testing samples can be categorized into simple and complex groups based on whether they have cells spanning multiple columns and rows. Notably, for the evaluation of detection performance, we use the formulation defined in Eq. (10), which is also the problem definition of PubTables1M (Smock et al., 2022), and the results generated by our single-label detection can be easily transformed into the multi-label detection results defined by PubTables1M (Smock et al., 2022), as discussed in Section 4.1.

To verify the effectiveness of our proposed solution, we include three state-of-the-art detection-based methods as benchmarks, including Cascade R-CNN (Cai & Vasconcelos, 2018), Deformable-DETR (Zhu et al., 2021) and Sparse R-CNN (Sun, Zhang et al., 2021), in which Cascade R-CNN (Cai & Vasconcelos, 2018) is also the based model of the proposed methods, Deformable-DETR and Sparse R-CNN are two state-of-the-art transformer-based detection models.

As discussed in Sections 1 Introduction, 2 Related work, image-to-sequence and graph-based models are another two types of solutions for the TSR problem. Therefore, we compare our proposed solution with these two types of solutions in this section. Specifically, for the image-to-sequence models, EDD (Zhong et al., 2020), TableFormer (Nassar et al., 2022), TableMaster (Ye et al., 2021), VAST (Huang et al., 2023) and MTL-TabNet (Ly & Takasu, 2023) are included. TSRFormer-DQ-DETR (Wang, Lin et al., 2023) and RobustTabNet (Ma et al., 2023) are two state-of-the-art models following the pipeline of detecting separation lines and then merging cell grids, which can be treated as a graph-based model as discussed in Section 2. TSRNet (Li, Yin et al., 2022) is also a graph-based methods which detect table cells first, then applies GNN to build the relations among the detected cells. Since FinTabNet and PubTabNet are the most widely used datasets for these non-detection-based models, we report the experimental results of these two datasets in Table 6, Table 7. It is worth mentioning that the PubTabNet dataset does not provide original PDF files, making it hard to generate detection annotations. Therefore, the model performance reported in Table 7 is calculated using the model trained with the PubTable1M dataset. Although the PubTable1M and PubTabNet datasets have misalignments regarding the ground truth HTML sequences, the proposed method still shows competitive performance compared with other state-of-the-art methods, as shown in Table 7. Our proposed method can also outperform non-detection-based models regarding structure-only TEDS on the FinTabNet dataset, as shown in Table 6.

Despite the superior performance of the FinTabNet and PubTabNet datasets, the proposed detection-based solution has some inherent limitations compared to the other two types of solutions. First, it is not suitable for dealing with tables with irregular shapes, such as distorted tables, because the predicted boxes of a typical object detection model are rectangles. Second, a detection-based model needs to work with an OCR tool to extract the text content from a table image, while image-to-sequence models usually combine the text extraction task with the TSR task. Therefore, our proposed solution can be better than other types of solutions when tables are well-formatted without distortions and OCR tools or PDF parsing libraries can easily extract their text content.

In this section, we conduct experiments on the FinTabNet dataset to demonstrate the effectiveness of our applied methods, including using the proposed single-label detection formulation, tuning parameters of RPN, applying the deformable convolution and spatial attention. It is worth mentioning that tuning parameters of RPN includes increasing the number of proposals and adjusting the aspect rations. Since other studies have successfully applied the effectiveness of increasing the number of proposals, we applied it to both the Cascade R-CNN baseline and the proposed TSRDet, as discussed in Sections 3.3 Revisiting region proposal generation, 5.2 Implementation details and experimental results. Therefore, we only discuss the impact of adjusting the aspect ratios for tuning parameters of RPN in this section.

The experimental results are shown in Table 8, Table 9, in which Asp_Ratio Tuning, Single_Label, DEFORM, and S_Attn are shorts for applying aspect ratio tuning, single label formulation, deformable convolution, and spatial attention, respectively. Even though Cascade R-CNN baseline can reach 95.06% regarding the mAP, its overall structure-only TEDS only reaches 82.70%. After tuning the aspect ratios for the anchor generation, the structure-only TEDS is increased to 90.23%, even though the mAP is only increased from 95.06% to 95.54%. Applying deformable convolution without other methods can improve the detection performance significantly but lead to a worse structure-only TEDS if we compare Ablation 1 and the Cascade R-CNN baseline. Ablation 3 and Ablation 4 show that transforming the multi-label detection formulation into single-label formulation can significantly improve the performance, and also make deformable convolution improve the model performance. And applying both deformable convolution and spatial attention together can further improve the model performance from the results of Ablation 4, 5 and TSRDet, as shown in Table 8. On the other hand, when it comes to detection metrics, applying deformable convolution always brings performance improvements from the results of Ablation 1 and Ablation 4, which can verify our analysis on the mismatch of detection metrics and cell-level metrics in Section 3.4.

As discussed in Sections 3.1 Preliminaries, 3.2 Rethinking problem formulations, multi-label detection tasks are difficult for two-stage detection models, but transformer-based detection models with learnable proposals can deal with multi-label detection tasks. Besides, the problem formulation of PubTables1M is a multi-label task, making it difficult for two-stage detection models. The experimental results from Section 5.2 can demonstrate our analysis. For example, as shown in Table 4, on the SciTSR dataset, the performance of Deformable-DETR and Sparse R-CNN are 97.30% and 98.30% regarding the structure-only TEDS, which are very close to the performance of proposed TSRDet (98.41%) and far better than the Cascade R-CNN baseline (79.09%). Notably, as mentioned in Section 5, all the models, except our proposed TSRDet, are using the multi-label detection setting. Therefore, two transformer-based detection models show promising results in the multi-label detection setting. Similarly, the experiments on the FinTabNet and PubTables1M datasets also show similar results. For example, the structure-only TEDS performance of Sparse R-CNN, TSRDet, and Cascade R-CNN baseline are 97.72%, 98.55%, and 83.78% on the PubTables1M dataset, 98.13, 99.05, and 87.49 on the FinTabNet dataset.

The experimental results in Sections 5.2 Implementation details and experimental results, 5.4 Ablation study show the misalignment of COCO and TEDS metrics many times. For example, in Table 5, both the Deformable-DETR and our proposed TSRDet can reach 96.28% regarding mAP on the SciTSR dataset, which is better than that of Sparse R-CNN. However, when it comes to structure-only TEDS, as shown in Table 4, both TSRDet and Sparse R-CNN can perform better than Deformable-DETR on the SciTSR dataset. Similar results also appear in the experiments on the FinTabNet dataset. As shown in Table 5, the mAP of Sparse-RCNN and Deformable-DETR are 96.38% and 96.68%, while their structure-only TEDS are 97.81% and 98.13%. More similar results can be found in the results of the ablation study, such as Ablation 1 and Ablation 3, as shown in Table 8, Table 9. To further verify our discussion in Section 3.4, we show the prediction results of Ablation 1 and 3 in Fig. 11. As discussed in Section 3.4, COCO metrics are relied on IoU scores, while TEDS is not. Therefore, as shown in Fig. 11(a), Ablation 1 with deformable convolution can better fit the extra white areas to improve the detection performance, but it cannot improve the TEDS compared with Ablation 3 whose result is shown in Fig. 11(b). It is worth mentioning that the ground truth of the sample in Fig. 11 has been shown in Fig. 6.

As discussed in Sections 3.4 Rethinking detection and TSR metrics, 3.5 Rethinking feature extraction, both generating good local features and building long-range dependencies are essential for a detection-based TSR model, and deformable convolution can improve the local feature generation but has the risk leading to the over-optimization to the detection performance. The ablation study’s experimental results can somewhat demonstrate our analysis. Considering the performance of Ablation 1 with deformable convolution in Table 8, Table 9, its TEDS is 84.35%, lower than the Cascade R-CNN baseline (87.49%), but its mAP is improved from 95.23% to 97.22%. These results not only show the misalignment of COCO and TEDS metrics but also demonstrate that merely improving local features can make the model fit empty spaces better, as shown in Fig. 11(a), but does not help alleviate the multi-label detection issue. Therefore, deformable convolution needs to be applied with other methods. On the other hand, our proposed Spatial Attention Module can improve the mAP and structure-only TEDS simultaneously if we compare the performance of Ablation 3 and 4, and also can be used with deformable convolution together to improve the structure-only TEDS further, as shown in Table 8, demonstrating the effectiveness of building long-range dependencies.

In this section, we conduct extra experiments in a cross-dataset setting to explore the generalization capacities of the trained models. Specifically, since we mainly use SciTSR, FinTabNet and PubTables1M datasets in the previous sections, in this section, we set up a cross-dataset setting using the training set from one of these three datasets to train the model and the testing sets of the rest of datasets to evaluate the model. As shown in Table 10, all three models show significant performance degradation in the cross-dataset setting, which is caused by the domain gap among the three datasets. On the other hand, the models trained with the FinTabNet and PubTables1M datasets show promising performance on the SciTSR testing set, especially for the model trained with the FinTabNet dataset. Moreover, the model trained with the FinTabNet dataset also performs better on the PubTables1M testing set than the model trained with the SciTSR dataset. Considering the scale of these datasets, as mentioned in Table 2, the diversity of samples and the model performance reported in Table 10, we conclude that the model trained with the SciTSR dataset has the worst generalization capacities. The model trained with the FinTabNet dataset can show better generalization capacities than the model trained with the PubTables1M dataset, even though PubTables1M is much larger.

In this section, we visualize some failed cases from the FinTabNet testing set and discuss possible underlying reasons. The model used in this section is trained and tested with the FinTabNet dataset. As discussed in previous sections, even though our proposed method can achieve promising results on the FinTabNet dataset, it can sometimes fail in some edge cases. For example, as shown in Fig. 12, the failed prediction is caused by the missing prediction of the Column Header, whose Column Header is a special case that lies out of the table box, as shown in Fig. 12(b). Fig. 13 shows another example which failed to predict the data cells. For this example, as shown in Fig. 13(b), the appearance of these data cells is texted in paragraphs, which is also different from common cases. Fig. 14 shows another edge case which classifies a regular row as a projected row header. This regular row is divided into two lines, making its appearance very similar to the projected row header. Besides these examples from the FinTabNet dataset, we include more cases in the Appendix. From the visualization of failed cases, we conclude that even with promising structure-only TEDS, we still need to pay more attention to the edge cases when applying our proposed models.

Besides the observations discussed in previous sections, the experimental results also show other phenomena that can be helpful in our model design. One observation is that Cascade R-CNN has better detection performance on small objects than Sparse R-CNN. For example, on the FinTabNet dataset, $APs$ of Sparse R-CNN is only 62.11%, while the Cascade R-CNN baseline and our proposed TSRDet reach 87.32% and 91.60%. This phenomenon might be caused by their methods of generating regional proposals. As discussed in Section 3.1, Cascade R-CNN uses RPN to generate regional proposals, which regress and classify anchor boxes, and the anchor boxes are generated by sliding the pre-defined boxes with different aspect ratios and sizes on the feature map of multiple scales. Therefore, Cascade R-CNN uses a dense proposal generation method (Sun, Zhang et al., 2021) with more region proposals, meaning that Cascade R-CNN can use the parameters of RPN to generate more high-quality small region proposals. By contrast, Sparse R-CNN uses sparse learnable regional proposals to replace dense proposals generated by the RPN, which can avoid parameter tuning of RPN but limit its performance on small objects. Another interesting observation is that the baseline Cascade R-CNN can work better on complex tables than simple tables, which is very different from other benchmark models. This phenomenon is caused by the fact that the spanning cells in complex tables are usually in the Column Row Headers, which can alleviate the multi-label detection issue. For example, Fig. 4, Fig. 4 show two samples from PubTables1M dataset, in which the former is a complex table and the latter is a simple table. Because of the existence of Spanning Cells in Fig. 4(c), the Column Header does not share its bounding box with any rows, which avoids multi-label detection. By contrast, the sample in Fig. 4(d) does not contain any Spanning Cell, making its Column Header share its bounding box with a Row, which is the challenging multi-label detection to Cascade R-CNN. As comparisons, Deforamble-DETR and Sparse R-CNN can deal with multi-label detection, and their performance on simple tables is better than complex tables regarding the structure-only TEDS, as shown in Table 4.

In this section, we summarize the key insights and critical design aspects for a detection-based TSR model. It is worth mentioning that the rationale behind these insights, along with the experiments validating them, has been thoroughly discussed in Sections 3, 5, and 6. First, a detection-based TSR solution needs to define the target table components properly to provide full table structural information. Some studies (Fernandes et al., 2023, Hashmi et al., 2021, Siddiqui et al., 2019, Xiao, Akkaya et al., 2022) over-simply the target detection components without Headers and Projected Row Headers, making them cannot fully recover the complex table structures. Furthermore, the problem formulation should align with the capacities of the employed detection model. For instance, PubTable1M (Smock et al., 2022) defines six types of target table components to fully reconstruct the complex table structures. However, it presents a multi-label detection definition, posing challenges for two-stage detection models. Therefore, in this study, we further develop the formulation of PubTable1M by introducing a pseudo-class to transform multi-label detection to regular single-label detection, as discussed in Section 4.1. Thirdly, existing studies usually employ COCO metrics to evaluate the detection-based TSR models. However, COCO metrics are insufficient for evaluating the TSR models, because ground truth boxes are often exceed the minimum bounding boxes required for capturing table structures, as discussed in Section 3.4, and models may optimize towards accommodating easier components with additional spaces, rather than effectively identifying challenging components. Hence, this study incorporates structure-only TEDS for model evaluation and introduces a Spatial Attention Module. The module is designed to establish long-range dependencies, enhancing the model’s ability to explore and address challenging components effectively. Fourthly, two-stage and transformer-based detection models have different capacities in the context of the TSR task. In this study, we leverage Cascade R-CNN and Sparse R-CNN as illustrative examples to highlight their differing capacities. More specifically, Sparse R-CNN excels in handling multi-label detection tasks without the need for tuning the region proposals because of its utilization of sparse learnable proposals, as discussed in Sections 3.1 Preliminaries, 3.3 Revisiting region proposal generation. By contrast, Cascade R-CNN cannot deal with multi-label detection tasks and needs to carefully tune the parameters of proposal generation because of the aspect ratios of defined table components, as discussed in Sections 3.1, 3.2 and 3.3. Additionally, Cascade R-CNN demonstrates superior performance on small objects compared to Sparse R-CNN, partially attributed to its dense and tunable proposal generation, as illustrated in Section 5.2. At last, while enhancing local feature extraction, such as employing deformable convolution, often leads to improved detection performance, it may not necessarily translate to enhanced TSR performance. It is necessary to build long-range dependencies, as discussed in Section 3.5. To sum up, it is imperative to ensure proper alignment between the problem formulation, capacities of detection models, evaluation metrics, and feature extraction in the context of a detection-based TSR solution. Our proposed Cascade R-CNN can be a demonstrative application of these insights in designing an effective detection-based TSR model.